168 research outputs found
Location-Aided Fast Distributed Consensus in Wireless Networks
Existing works on distributed consensus explore linear iterations based on
reversible Markov chains, which contribute to the slow convergence of the
algorithms. It has been observed that by overcoming the diffusive behavior of
reversible chains, certain nonreversible chains lifted from reversible ones mix
substantially faster than the original chains. In this paper, we investigate
the idea of accelerating distributed consensus via lifting Markov chains, and
propose a class of Location-Aided Distributed Averaging (LADA) algorithms for
wireless networks, where nodes' coarse location information is used to
construct nonreversible chains that facilitate distributed computing and
cooperative processing. First, two general pseudo-algorithms are presented to
illustrate the notion of distributed averaging through chain-lifting. These
pseudo-algorithms are then respectively instantiated through one LADA algorithm
on grid networks, and one on general wireless networks. For a grid
network, the proposed LADA algorithm achieves an -averaging time of
. Based on this algorithm, in a wireless network with
transmission range , an -averaging time of
can be attained through a centralized algorithm.
Subsequently, we present a fully-distributed LADA algorithm for wireless
networks, which utilizes only the direction information of neighbors to
construct nonreversible chains. It is shown that this distributed LADA
algorithm achieves the same scaling law in averaging time as the centralized
scheme. Finally, we propose a cluster-based LADA (C-LADA) algorithm, which,
requiring no central coordination, provides the additional benefit of reduced
message complexity compared with the distributed LADA algorithm.Comment: 44 pages, 14 figures. Submitted to IEEE Transactions on Information
Theor
Decentralized Differentially Private Without-Replacement Stochastic Gradient Descent
While machine learning has achieved remarkable results in a wide variety of
domains, the training of models often requires large datasets that may need to
be collected from different individuals. As sensitive information may be
contained in the individual's dataset, sharing training data may lead to severe
privacy concerns. Therefore, there is a compelling need to develop
privacy-aware machine learning methods, for which one effective approach is to
leverage the generic framework of differential privacy. Considering that
stochastic gradient descent (SGD) is one of the mostly adopted methods for
large-scale machine learning problems, two decentralized differentially private
SGD algorithms are proposed in this work. Particularly, we focus on SGD without
replacement due to its favorable structure for practical implementation. In
addition, both privacy and convergence analysis are provided for the proposed
algorithms. Finally, extensive experiments are performed to verify the
theoretical results and demonstrate the effectiveness of the proposed
algorithms
- …